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Abstract. We introduce the notion of a weakly reflective submanifold, which 
is an austere submanifold with a certain global condition, and study its fun- 
damental properties. Using these, we determine weakly reflective orbits and 
austere orbits of s-rcprcscntations. 



1. Introduction 

Orbits of an s-representation, that is a linear isotropy representation of a Rie- 
mannian symmetric pair, are important examples of homogeneous submanifolds in 
the hypersphere of a Euclidean space. For example, a homogeneous isoparametric 
hypersurface in the hypersphere, which many mathemtaticians have investigated, 
can be obtained as a principal orbit of an s-representation of a Riemannian sym- 
metric pair of rank two. The family of isoparametric hypersurfaces has a unique 
minimal isoparametric hypersurface. Furtheremore, typical examples of minimal 
submanifolds in the hypersphere are given as orbits of s-representations. Hirohashi- 
Song-Takagi-Tasaki |7J showed that there exists a unique minimal orbit in each 
strata of the stratification of orbit types. However, in general we can not explicitly 
point out which orbit among each strata is a minimal submanifold. 

Harvcy-Lawson [4] introduced the notion of an austere submanifold, which is 
a minimal submanifold whose second fundamental form has a certain symmetry. 
They showed that one can construct a special Lagrangian cone, therefore absolutely 
area-minimizing, in a complex Euclidean space as the twisted normal bundle of an 
austere submanifold in a sphere (see [4], [2]). As we mentioned above, the com- 
plete list of minimal orbits of s-representations in the hypersphere is unknown at 
the moment. Therefore we first attempt to determine all austere orbits. We give 
a necessary and sufficient condition for an orbit to be an austere submanifold in 
the hypersphere in terms of the restricted root system of a Riemannian symmetric 
pair. By this criterion, we can determine all orbits which are austere submanifolds 
in the hypersphere. Since the definition is focused on a symmetry of its second 
fundamental form, the notion of an austere submanifold is an infinitesimal prop- 
erty of a submanifold. However, we observe that some of austere orbits, which we 
classified, have a certain global symmetry. This symmetry is a globalization of the 
notion of an austere submanifold and a weakened condition of a reflective subman- 
ifold. Therefore we shall call them weakly reflective submanifolds, and study some 
fundamental properties of them. Finally we determine all weakly reflective orbits 
of s-representations. 
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The organization of this paper is as follows. In Section 2, we will give the 
definition of a weakly reflective submanifold (Definition ^. and recall some related 
notions. We study their relationship and fundamental properties. In Section 3, we 
summarize geometry of orbits of s-representations of Riemannian symmetric pairs. 
This will be a preliminary for the sections below. In Section 4, we shall give 
the list of orbits of s-representations which are weakly reflective submanifolds in 
the hypersphere (Theorem 14. ip . We show that these orbits are weakly reflective 
submanifold in the hypersphere there, however, we will show that the list gives 
all weakly reflective orbits later. In Section 5, we will give a criterion of austere 
orbits (Lemma 15 ,3[) , and determine all orbits which are austere submanifolds in 
the hypersphere (Theorem 15. ip . Furthermore we show that austere orbits which 
are not enumerated in the list of weakly reflective orbits are not weakly reflective 
submanifolds. Then we will complete the proof of the list of weakly reflective orbits. 
In Section 6, we will study relationships between weakly reflective submanifolds in 
a sphere and those in Euclidean spaces or complex projective spaces. 

The authors are profoundly grateful to Makoto Kimura and Osami Yasukura for 
their helpful suggestion on Proposition 14.31 Before we wrote this paper, Kimura, 
Yasukura and the third named author showed a previous version of Proposition 
14.31 which is unpublished, that is, the orbit of the highest root of a compact Lie 
group under the adjoint action is an austere submanifold in the hypersphere. The 
authors would also like to thank Reiko Miyaoka for her valuable comments. In fact, 
Proposition 12 . 91 was essentially suggested by her. 

2. Definitions and fundamental results 

We begin with recalling the definition of a reflective submanifold given by Leung 
[8]. Let M be a complete Riemannian manifold. A connected component of the 
fixed point set of an involutive isometry of M is called a reflective submanifold. A 
reflective submanifold is a complete totally geodesic submanifold. The involutive 
isometry which defines a reflective submanifold M can be determined uniquely. We 
call it the reflection of M and denote by o~m- If M is a reflective submanifold in 
M and <7m is its reflection, then for any normal vector £ £ T^M 

a M (x) = X, {da M )xi = cr M (M) = M 

hold. Taking notice of these properties, we define a weakly reflective submanifold 
as follows. 

Definition 2.1. Let M be a submanifold of a Riemannian manifold M. For each 
normal vector £ E T^M at each point x £ M, if there exists an isometry of M 
which satisfies 

ae(x) = x, (daz)^ = ir c (M) = M, 

then we call M a weakly reflective submanifold and a reflection of M with respect 
to £. 

In the case where M is a hypersurface, is independent of the choice of £ at 
each point x. In this paper mainly we deal with orbits of some isometric actions of 
compact Lie groups. We note that if M is an extrinsic homogeneous submanifold 
in M, that is an orbit of an isometric action of a Lie group on M, then it suffices 
to show the condition to be a weakly reflective submanifold at one point of M. 
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Remark 2.2. For a reflective submanifold, there exists a reflection which is in- 
dependent of the choice of a normal vector. So it is clear that the definition of a 
weakly reflective submanifold is a weakened condition of a reflective submanifold. 

Example 2.3. 

S n ~ l (l) x S n -\1) = {(x,y) | x,ye S n ~ l (l)} 

is a weakly reflective submanifold in (2n — 1)- dimensional sphere S 2n ~ 1 {-\/2) of 
radius y/2. 

Proof. Since 5" _1 (1) x S'" _1 (l) is a homogeneous submanifold of S 2n ~ 1 (V*)i it 
suffices to show the condition to be weakly reflective at one point of S n ~ 1 (l) x 
S^il). The tangent space of S 71 ' 1 ^) x S 11 ' 1 ^) at 

n + l 

x = (i,o,...,o, T ,o,...,o) g s^-^i) x s™- 1 ^) 

is given by 

T^-Hl) x S n - 1 {l))={(0,x 2 ,...,x n ,0,V2,...,Vn)\xi,Vj G R}, 
and the normal space in 5' 2n_1 (v / 2) is 

n+l 

r^^-^i) x s™- 1 ^)) = R(i,o, . . . ,o, -i,o, . . . ,o). 

Now we define an isometry a of S 2n ~ 1 {-J 7 2) by 

a(xi, ...,x n ,yi,...,y„) = (y%, ...,y n ,xi,.. .,x n ) 

for (an, ...,x n , yi ,.. .,y n ) G S 2 "" 1 ^)- Then 

<t(;b) - s, ^(^"-^l) x S"- 1 ^)) = S*™- 1 ^) x S^l) 

and da x acts on T^S"- 1 ^) x S™" 1 ^)) as -id. Thus S"- 1 ^) * ^""H 1 ) is a 
weakly reflective submanifold in S 2n ~ 1 (y/2), □ 

Definition 2.4. Let M be a submanifold of a Riemannian manifold M. We denote 
the shape operator of M by A. M is called an austere submanifold if for each normal 
vector £ G T^rM 1 the set of eigenvalues of Aj is invariant (concerning multiplicities) 
under multiplication by —1. It is obvious that an austere submanifold is a minimal 
submanifold. 

The notion of an austere submanifold was first given by Harvey-Lawson [4] . 

Proposition 2.5. A weakly reflective submanifold is an austere submanifold. 

Proof. Let M be a weakly reflective submanifold in a Riemannian manifold M. 
Then for each normal vector £ G T^M, there exists an isometry o~£ of M which 
satisfies 

a^(x) = x, (du*) s £ = <t*(M) = M. 

For a normal vector £ G T^M, we denote by the shape operator of M with 
respect to £ and by h the second fundamental form of M. For 1,7 £ T X M, we 
take vector fields X and F defined on a neighborhood of x in M which are tangent 
to M and X. x = X and Y x = Y. Since satisfies a^(M) = M, vector fields da^X 



4 



OSAMU IKAWA, TAKASHI SAKAI, AND HIROYUKI TASAKI 



and d<j(Y are tangent to M. Let V denote the covariant derivative of M. Then we 
have 

h((dat) x X,(das) x Y) = (V^d^Y^ = ((da^V^Y ) ± 
= (d*t) x (y g Y) ± = (d*t) x h{X,Y). 

From 

(As(da ( ) x X,(das) x Y) = (h((da ( ) x X, (d^) x Y), £) 

= ((das) x h(X, Y), = (h(X, Y), (da^O 
= (h(X,Y),-0=-{A i X,Y), 

we have (dcr^)j 1 A^(da^) x — —A^. This implies that (da^) x provides an isomor- 
phism between eigenspaces of A^ for eigenvalues A and —A. Thus M is an austere 
submanifold. □ 

In the rest of this section, we shall study weakly reflective orbits of isometric 
actions of Lie groups on Riemannian manifolds. First we shall provide some pre- 
liminaries. Let G be a Lie group acting isometrically on a Riemannian manifold 
M and G x be the isotropy subgroup at x, that is, G x = {g £ G \ gx — x}. Then 
the orbit G(x) is diffeomorphic to the coset manifold G/G x . An orbit G(x) is a 
principal orbit if, for any y £ M, there exists g £ G such that G x C gGyg^ 1 . It is 
known that there exists a principal orbit. The codimension of a principal orbit is 
called the cohomogeneity of the action of G on M. An orbit which is not principal 
is called a singular orbit. The differential of the action of G x defines a linear rep- 
resentation of G x on T X M called the linear isotropy representation. The tangent 
space T x (G(x)) and the normal space T x L (G(x)) of G(x) at x are invariant sub- 
spaces of the linear isotropy representation. The restriction of the linear isotropy 
representation to T x L (G(x)) is called the slice representation at x. 

Theorem 2.6. (Slice representation theorem [9J Theorem 1.1], [TTl Theorem 4.6], 
[12( Proposition 5.4.7]). The cohomogeneity of a slice representation equals the 
cohomogeneity of the action of G on M . Moreover, G(x) is a principal orbit if and 
only if the slice representation at x is trivial. 

Proposition 2.7. Any singular orbit of a cohomogeneity one action on a Rie- 
mannian manifold is a weakly reflective submanifold. 

Proof. Suppose that the isometric action of a Lie group G on a Riemannian manifold 
M is cohomogeneity one. Let G(x) be a singular orbit. 

First we consider the case where the codimension of G(x) is equal or greater 
than 2. From the slice representation theorem, the isotropy subgroup G x acts 
transitively on the hypersphere in T x (G{x)). In particular, for any £ £ T x {G(x)) 
there exists g £ G x such that dg x (^) = — £. Therefore g becomes a reflection of 
G(x) at x with respect to £. Since G{x) is a homogeneous submanifold, G(x) has 
a reflection with respect to any normal vector at any point. Thus G{x) is a weakly 
reflective submanifold in M. 

When the codimension of G{x) is 1, the slice representation at x is not trivial. 
Therefore for any £ 6 T^{G(x)) there exists g £ G x such that dg x (^) = — £. 
Thus, by the same discussion with above, G(x) is a weakly reflective submanifold 
in M. □ 
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Remark 2.8. Podesta [13] proved that any singular orbit of a cohomogeneity one 
action is an austere submanifold. However, essentially he showed Proposition 12.71 

Proposition 2.9. Let G be a connected Lie group acting isometrically on a com- 
plete, connected Riemannian manifold M . Suppose that the action of G on M is 
cohomogeneity one with two singular orbits. If there exists a principal orbit which is 
a weakly reflective submanifold in M , then it has a same distance from two singular 
orbits and two singular orbits are isometric. 

Proof. Since there exist two singular orbits, the orbit space M/G is homeomorphic 
to a closed interval (Mostert [10] . Bergery p]). Orbits of interior points are principal 
and those of end points are singular. Moreover principal orbits are hypersurfaces 
in M, because M/G is homeomorphic to a closed interval. Suppose that G(x) 
is a principal orbit which is a weakly reflective submanifold. Then, by the slice 
representation theorem, there exists a unit normal vector field £ on G(x), which is 
invariant under the action of G. We take a geodesic j(t) of M which satisfies an 
initial condition 

7 (0) = x, 7'(0)=£x. 
Then j(t) is a section of the action of G on M. Since dg x ((; x ) — ^ gx for any g G G, 
gj(t) is a geodesic of M which satisfies an initial condition 

57 (0) = gx, ( 57 )'(0) = £ gx . 

Since G(x) is a weakly reflective submanifold of M, there exists an isometry a of 
M which satisfies 

a(x) = x, do- x (£ x ) = -( x , o-{G(x)) = G(x), 

that is a reflection of G{x) with respect to £ x . We set 

G(x) ± ={ye G(x) | da v {£ v ) = ±& (y )}. 

The sets G(x) + and G(x)_ are closed subsets of G(x), and G(x) is a disjoint union 
of G(x)+ and G(x)_ because G{x) is a hypersurface in M . Since G(x) is connected 
and x G G(x)_, we have G{x) — G(x)-. This implies that da y (^ y ) — —£ a (y) f° r anv 
y G G(x). For any jgG, ag^/(t) is a geodesic which satisfies an initial condition 

cr 57 (0) = a(gx), (o-gj)'(O) = da gx (gj)'(0) = da gx (^ gx ) = -£ a g X - 

Now we take g\ G G such that g\x — a(gx). Then o~gj(t) and gij{—t) are geodesies 
of same initial conditions, hence crg"f(t) = <7i7(— t) G G("f(—t)). Therefore we have 
a(G(^(t))) C G(7(— t)) for each t. Since cr _1 is also a reflection of G(x) at x, we also 
have cr -1 (G(7(— t))) C G(7(t)) by the same discussion for <r -1 and ~f(—t). Thus 
a(G(^(t))) — G(7(— t)). This implies that two singular orbits can be expressed as 
G(7(£i)) and G(7(— for some t\. Consequently we have the conclusion. □ 

3. Orbits of s-representations 

A linear isotropy representation of a Riemannian symmetric pair is called an 
s-representation as we mentioned in Introduction. In the following sections, we will 
study orbits of s-representations which are austere submanifolds and weakly reflec- 
tive submanifolds. For this purpose, we shall provide some fundamental notions of 
orbits of s-representations in this section. 
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Let G be a compact, connected Lie group and K a closed subgroup of G. Assume 
that 9 is an involutive automorphism of G and Gg C K C G$, where 

G = {g e G | %) = 5 } 

and is the identity component of Gg . Then (G, K) is a symmetric pair with 
respect to 9. We denote the Lie algebras of G and K by g and fi, respectively. The 
involutive automorphism of g induced from will be also denoted by 9. Then we 
have 

t = {X eg\9(X)=X}. 

Take an inner product ( , ) on g which is invariant under 9 and the adjoint repre- 
sentation of G. Set 

m = {X eg\0(X) = -X}, 
then we have a canonical orthogonal direct sum decomposition 

fl = t + m. 

Henceforth we assume that the symmetric pair (G, K) is irreducible, namely K acts 
irreducibly on m. 

Fix a maximal Abelian subspace o in m and a maximal Abelian subalgebra t in 
g containing a. For a £ t we set 

fl a - {X e fl c I [ff, A] = >/=T<a, # }A (# g t)} 

and define the root system i? of g by 

i?-{aet-{0}|g Q ^{0}}. 

For a e a we set 

0a = {A e g c | [ff, A] = V^T(a, ff)A (if e a)} 
and define the restricted root system R of (g,fi) by 

R = {aea-{O}\ 0a ^{O}}. 

Set 

and denote the orthogonal projection from t to a by H i— > _ff . Then we have 

i? = {a | a £ R- R }. 

We take a basis of t extended from a basis of a and define the lexicographic orderings 
> on o and t with respect to these bases. Then for H £ t, H > implies H > 0. 
We denote by F the fundamental system of R with respect to the ordering >. Set 

F =Fni?o, 

then the fundamental system F of R with respect to the ordering > is given by 

F = {a | a £ F — F }. 

We set 

R+ = {a £ R | a > 0}, R+ = {a £ R | a > 0}. 

Then we have 

i? + = {a | a £ R + - R }. 

We also set 

6 = {Aefi| [A, H] = (H £ a)}, 
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and define 

6 a =6H (fla+fl-c), m a =mn(9 a +j- a ) 
for a G R + . Under these notations, we have the following lemma. 

Lemma 3.1 ( 14J ) . (1) We have orthogonal direct sum decompositions 
6 = «o+ la, m = a+ ^2 m a . 

(2) For each a G i?+ — Ro, there exist S a G t and T a G m such that 
{S a | a G R+, a — A}, {T a \ a G R+, a = A} 
are respectively orthonormal bases of t\ and m\ and that for H G a 

[fT,S a ] = <a,.ff)T ai [H,T a ] = -{a,H)S a , [S a ,T a ] = a, 
Ad(expH)S a = cos(a, H)S a + sin(a, H)T a , 
Ad(expH)T a = - sin(a, J2)S Q + cos(a, H)T a . 

We define a subset D of a by 

D= \J{H €a \ (a, H) =0}. 

A connected component of a — D is a Weyl chamber. We set 

C = {H G a | (a, H) > (a G F)}. 
Then C is an open convex subset of a and the closure of C is given by 

C = {H G a | (a, H) > (a G F)}. 
For a subset A C F, we define 

C A = {H G (5 | (a, H) > (a G A), {fi, H) = (0 G F - A)}. 
Lemma 3.2. (1) For Ai C F, </ie decomposition 

C^ = \J C A 

AcAi 

is a disjoint union. In particular, C = C A is a disjoint union. 

AcF 

(2) For Ai, A 2 C F, Ai C A 2 i/ and or% i/ C Al C (7 s *. 
For each a G F, we take if a G o such that 



Then we have 



and for A C F 



c — I 2_j taH a 

{ a<EF 
C A = < 22 taH a 



t a >o 



t a >0 



. aeA 
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We set 



R A = i?n(F-A) z , 
R^t = R A n R+, 



We also set 



e A = g A nt = h+ ^, 

a£Rf 

m A = g A n m = a + ^ m a . 



Then we have an orthogonal direct sum decomposition 

fl A =t A +m A . 

For H £ m we set 

Z% = {k 6 K | Ad(fc)ff = ff}. 

Then is a closed subgroup of K and the orbit Ad(K)H is diffeomorphic to the 
coset manifold K/Z^- 

Under these notations, we have the following lemma. 

Lemma 3.3 ( 6 ). Fix a subset A C F. For H 6 C A we have the following: 

(1) R A = {aeR+\ (a,H)=0}, 

(2) R A = {a e R\ (a,H) = 0}, 

(3) q a = {X e B \{H,X} = 0}, 

(4) (g A ,fi A ) is a symmetric pair and its canonical decomposition is given by 

A =t A + m A , 

(5) 6 A is the Lie algebra of Z^- 

Now we shall study an orbit Ad(K)H of the linear isotropy representation of 
(G,K) through H G m. An orbit Ad(K)H is a submanifold of the hypersphere S 
of radius \\H\\ in m. From [6], Ad(i^)iJ is connected. Since 

m = |J Ad(fc)C, 

without loss of generalities we may assume H £ C. Moreover, from Lemma [3.21 
there exists AcF such that H 6 C A . For X e t we define a vector field A* on m 

by 

X*=Y Ad(exptA)ir = [A,x] 

at x <E m. Then A*|Ad(x)H is a tangent vector field on Ad(K)H. From Lemma l3~T1 
we have the following lemma. 



Lemma 3.4. For A C F and H e C A , the tangent space T H {Ad{K)H) of the 
orbit Ad{K)H at H and the normal space T^(Ad(K)H) in the hypersphere can be 
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expressed as 

T H {Ad{K)H) = m 



Oil 

aeR+-R$ 



T^(Ad(K)H) = H ± r\a+ m «= [j Ad(fc)(i7 ± n a). 

aER$ kez" 

Let h denote the second fundamental form of Ad(K)H at H in the hypersphere S. 
Then we have 

h(X* H ,Y£) = [Y,[X,H]} N , 
where [Y, [X,H]] N is T^(Ad(K)H)- component of [Y, [X,H]]. 

Proof. From Lemma 13.11 we have 

T H (M(K)H) = m «' 

aeR+-R.f 

T^{Ad{K)H) = H 1 - n o+ m « = H± nmA 

a£R$ 

Moreover, from Lemma 13.31 

m A = |J Ad(fc)o. 

kez" 

Since Ad(k)H = H for k € Z§, we have 

H ± nm A = [j Ad(fc)(i/ ± na). 

fcezf 

The calculation of X* mentioned above shows the representation of the second 
fundamental form. □ 

For orbits of s-representations which are minimal submanifolds in the hyper- 
sphere, the following theorem is known. 

Theorem 3.5 (7 ]). Fix a hypersphere 5mm centered at 0. For each subset A C F, 
there exists a unique H € C A n S such that Ad(K)H is a minimal submanifold in 
S. 

However, in general we can not determine H where Ad(K)H is a minimal sub- 
manifold in S explicitly. In the following two sections, we will give the complete 
lists of H where Ad(K)H is an austere submanifold and a weakly submanifold in 
S. 

4. Weakly reflective orbits of s-representations 

In this section, we shall study orbits of irreducible s-representations which are 
weakly reflective submanifolds in the hypersphere. In the next section, we will 
study austere orbits. Since these two properties of orbits are invariant under scalar 
multiples on the vector spaces, we do not discriminate the difference of the length 
of a vector. The following theorem is the main result of this section. We shall 
follow the notations of root systems in [3] . 

Theorem 4.1. An orbit of an irreducible s -representation which is a weakly reflec- 
tive submanifold in the hypersphere is one of the following list: 
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(1) an orbit through a restricted root vector (Proposition 14. 3j ). 

(2) the orbit through the vector 2e± — e2 — e% or e\ + C2 — 2e3 of the linear 
isotropy representation of a compact symmetric pair with restricted root 
system {±(ej — ey)} of type A2 (Proposition 14. 4j ). 

(3) the orbit through the vector e\ + e2 — e% — e^ of the linear isotropy represen- 
tation of a compact symmetric pair with restricted root system {±(ei — e^)} 
of type A3 (Proposition 14. 5p . 

(4) the orbit through the vector e\ of the linear isotropy representation of a 
compact symmetric pair with restricted root system {±(ej ± e^)} of type Di 
(Proposition 14. 6[) , 

(5) the orbit through the vector e% + C2 + e% ± e^ of the linear isotropy represen- 
tation of a compact symmetric pair with restricted root system {±(ei ± e^)} 
of type D4 (Proposition [47]). 

Here we prove that orbits listed above are weakly reflective submanifolds. In 
the next section, we will classify all austere orbits of irreducible s-representations 
and show that all weakly reflective orbits can be obtained in Theorem 14. 1 1 For this 
purpose, we shall first give the following lemma. 

Lemma 4.2. For H £ a, the orbit Ad(K)H is a weakly reflective submanifold in 
the hypersphere S if and only if for any £ G H n a, there exists a linear isometry 
<7£ of m which satisfies 

(4.1) ^(H)=H, ^(£)=-£, * 6 (M(K)H) = M(K)H. 

Proof. From Lemma [341 the normal space of the orbit Ad(K)H at H in S is given 

by 

Tff(Ad(K)H) =ff i nol^m„= (J Ad(fc)(iJ ± n o). 

If Ad(K)H is a weakly reflective submanifold in S, then for £ £ H 1 - fl a there exists 
a linear isometry of m which satisfies 

o- t {H) = H, {do- t )„ti = a ( (Ad(K)H) = Ad(K)H. 

Here we have (da^)n = eg, since o~£ is a linear isometry. 

Conversely, assume that Ad(K)H satisfies the condition (|4.1|) . We take an ar- 
bitrary normal vector £ 6 Tjg(Ad(K)H). From Lemma T3.4[ there exists fco G 
such that Ad(fco)£ € H (1 o. Then, from the assumption, there exists a linear 
isometry a which satisfies 

a(H) = H, oAd(fc )£ = -Ad(fco)f , a(Ad(K)H) = Ad(K)H. 

We now define = Ad(fco) _1 aAd(fco). Then satisfies 

a c (H) = H, a&) = a s (Ad(K)H) = Ad(K)H. 

Thus ctj is a reflection of Ad(K)H with respect to a normal vector £ at H . Since 
Ad(K)H is a homogeneous submanifold, we have a reflection with respect to any 
normal vector at arbitrary point. Consequently Ad(K)H is a weakly reflective 
submanifold in S. □ 

Proposition 4.3. An orbit through a restricted root vector of the linear isotropy 
representation of an irreducible compact symmetric pair is a weakly reflective sub- 
manifold in the hypersphere S. 
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Proof. Let ao be a restricted root vector and put H — ao- The reflection s ao on a 
with respect to ao is given by 

s ao {X)=X-- -a (X G a) 

(ao, ao) 

and satisfies 

s a (H) = —H, s ao \ anH ± = l anH ±. 
The reflection s ao is an element of the Weyl group, hence there exists fco G Nk 
such that Ad(fc )| Q = s ao , where 

N K = {ke K\ Ad(k)a = a}. 

Therefore 

-H = Ad(k )H e Ad(K)H, 
and we have Ad(K)(— H) = Ad(K)H. We define a linear isometry a of m by 

a = -Ad(fe )| ro . 

Then a satisfies 

a(H) = H, a\ anH ± = -l\ anH ±, a(Ad(K)H) = Ad(K)H. 
Thus, from Lemma \4. 21 Ad(K)H is a weakly reflective submanifold in S. □ 

Proposition 4.4. The orbit through the vector 2e± — e 2 — ^3 or e\ + e 2 — 2e3 of 

the linear isotropy representation of a compact symmetric pair with restricted root 
system {±(ei — ej)} of type is a weakly reflective submanifold in the hypersphere 
S. 

Proof. Since the symmetric pair (G, K) is of rank 2, the action of K on S is coho- 
mogeneity one. The vector 2ei — e 2 — e-i (resp. e± + e2 — 2es) is orthogonal to a 
restricted root e 2 — e% (resp. e\ — e 2 ). Therefore the orbit of K through 2ei — e 2 — &z 
(resp. ei + e2 — 2e3) is a singular orbit. Hence from Proposition 12. 71 this orbit is a 
weakly reflective submanifold in S. □ 

Proposition 4.5. The orbit through the vector ei + e2 — e^ — e 4 of the linear isotropy 
representation of a compact symmetric pair with restricted root system {±(ej — ej)} 
of type A 3 is a weakly reflective submanifold in the hypersphere S . 

Proof. Put = ei + 62 — e3 — £4. The set i?^ of all positive restricted roots which 
are orthogonal to H is given by 

R+ = {ei - e 2 , e 3 - e 4 }. 

Let s ei _ e2 and s e3 - e4 be the reflections with respect to restricted roots e\ — ei and 
e3 — e 4 , respectively. Then s ei _ e2 and s e3 _ e4 are elements of the Weyl group, hence 
there exist k ,ki G Nx such that 

S ei - e2 = Ad(/C )|o, Se 3 -e 4 = Ad(/ci)| . 

We now define a linear isometry of m by 

a{X) = Ad(fc )Ad(A:i)A (X G m). 

Then a satisfies 

a{H) = H, a\ anH ^ = -l anH x , a(Ad(K)H) = Ad(K)H. 
Thus from Lemma \A. 21 Ad(K)H is a weakly reflective submanifold in S. □ 
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Proposition 4.6. The orbit through the vector e\ of the linear isotropy represen- 
tation of a compact symmetric pair with restricted root system {iet ± ej} of type 
Di is a weakly reflective submanifold in the hyper sphere S . 

Proof. An irreducible compact symmetric pair with restricted root system of type 
Di is one of {SO {21) x SO {21), SO{2l)*) and {SO{2l),SO{l) x SO (I)). 

First we consider the case of (50(2/) x 50(2/), 50(2/)*). In this case, m can be 
identified with a {21) in a natural manner. We take a maximal Abelian subalgebra 



diag 





h 



o 





tl 

-1 









,ti e R 



,0,. 



of o(2/), and put 

H = ex — diag 

We define a linear isometry a of o{2l) by 

a{X) = sXs {X G o(2/)), 

where 

" i o i r -l o i r -l o 

1 J ' [ 1 J [ 1 

Then a is an isometry of 5 and satisfies 

a(H) = H, a\ anH , = -id anH x, a{kd{K)H) = Ad{K)H. 

Hence from Lemma \A. 21 Ad(!f )iJ is a weakly reflective submanifold in 5. 

Second, we consider the case of {SO {21), SO{l) x SO{l)). We take a maximal 
Abelian subspace 



diag 



€ 0{2l). 



and put 





-X 



H 



X 




ei 



X = diag(ti, . . .,U), U S R 







An 




where 



X = diag{l,0,. 
We define a linear isometry a of m by 

a{X) 



e o(2/), 
0} e M,(R). 

(A G m), 

» ji j |_ <j 
where 

S = diag{l,-l,...,-l}eO(Z). 
Then a is an isometry of 5 and satisfies 

a{H) = H, a\ anH ± - -id anH ±, a{Ad{K)H) = Ad{K)H. 
Hence from Lemma F4.2[ Xd{K)H is a weakly reflective submanifold in 5. 



" s " 




' s " 


. ^ . 


A 


. 7 * . 



□ 



Proposition 4.7. TTie or/n£ through the vector ei + e2+e3±e4 of the linear isotropy 
representation of a compact symmetric pair with restricted root system {±ei ± ej} 
of type Di is a weakly reflective submanifold in the hyper sphere 5. 
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Proof. We take a fundamental system of {±e^ ± ej} of type -D 4 : 

Q'i = ex - e 2 , a 2 = e 2 - e 3 , a 3 = e 3 - e 4 , a 4 = e 3 + e 4 . 

The automorphism group of the Dynkin diagram is the permutation group of 
{ai,a 3 ,a 4 }. So there exists an automorphism of m mapping u\ to a 4 and fix- 
ing a 3 , which gives an equivalence of the orbits through e 4 and e 4 + e 2 + e 3 + e 4 . 
Thus the orbit through e\ + e 2 + e 3 + e 4 is also a weakly reflective submanifold in 
the hypersphere. Similarly the permutaion of ot\ and a 3 gives an equivalence of the 
orbits through e\ and e\ + e 2 + e 3 — e 4 . Thus the orbit through e 4 + e 2 + e 3 — e 4 
is also a weakly reflective submanifold in the hypersphere. □ 

In the case of (50(8), 50(4) x 50(4)) we can explicitly represent a reflection of 
the orbit though e\ + e 2 + e 3 + e 4 . The linear isotropy representation is equivalent 
to 

( 9l ,g 2 ) ■ X = gi Xg^ ((<?!, 52) G 50(4) x 50(4), X G M 4 (R)). 

Let ej denotes an element of M 4 (R) whose component is 1 and others are 0. 
Then the orbit through e x + e 2 + e 3 + e 4 is 50(4) in M 4 (R). 
For z\ <E> z 2 £ H <£> H, we define 

Zl ® Z2 : H -> H ; z ^ z-izz 2 - 

Since H = R 4 , we can regard </> Zl ® Z2 as an element of M 4 (R) and <j> induces an 
isomorphism M 4 (R) = H®Hof real algebras. We define an involutive isometry a 
of H <g) H by 

cr:H(K)H^H(K)H; z\® z 2 ^ z\®z 2 - 
We also denote by a the linear isometry of M 4 (R) induced from a through <j>. We 
note that 

50(4) = {0 2l822 |z 1 ,z 2 e5p(l)}. 

Moreover 

{z x ® z 2 I zi,z 2 G 5p(l)} C H ® H 
is invariant under a. Therefore 50(4) is invariant under a. The identity element I 
is fixed by the action of a. The normal space of 50(4) at I in 5 15 (2) is given by 

T/(50(4)) = {X e M 4 (R) I X : symmetric, tr(X) = 0}. 

It is easy to see that 2^(50(4)) is contained in the eigenspace of a for an eigenvalue 
— 1. Thus a is a reflection of 50(4) with respect to an arbitrary normal vector at 
I. 

5. Austere orbits of s-representations 

In this section we classify all orbits of irreducible s-representations which are 
austere submanifolds in the hypersphere 5. In the previous section we showed 
that all orbits through a restricted root vector (or its scalar multiple) are weakly 
reflective, hence austere. Therefore, hereafter we shall concern with other orbits. 
We will also determine austere orbits which are not weakly reflective submanifolds. 
Then we will complete to prove Theorem 14.11 

The classification of austere orbits is following: 

Theorem 5.1. An orbit of an irreducible s -representation which is an austere 
submanifold in the hypersphere is one of the following list: 
(1) an orbit through a restricted root vector, 
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(2) the orbit through the vector 2ei — e 2 — ez or e\ + e2 — 2e% of the linear 
isotropy representation of a compact symmetric pair with restricted root 
system {±(ej — e^)} of type A 2 (Proposition 14.41 or 15. 4p . 

(3) the orbit through the vector e\ + e 2 — e% — e^ of the linear isotropy represen- 
tation of a compact symmetric pair with restricted root system {±(ei — e^)} 
of type A3 (Proposition 15. 4|) . 

(4) the orbit through the vector e\ of the linear isotropy representation of a 
compact symmetric pair with restricted root system {±(ei ± ej)} of type D[ 
(Proposition 15. 5p , 

(5) the orbit through the vector e\ + e 2 + e% ± e^ of the linear isotropy represen- 
tation of a compact symmetric pair with restricted root system {±(ei ± e^)} 
of type D4 (Proposition [53]), 

(6) the orbit through the vector e\ + ei ^ 2 of the linear isotropy representation 
of a compact symmetric pair with restricted root system {±ei,±ei ± ej} of 
type B 2 whose multiplicities are constant (Proposition 15. 6| ) . 

(7) the orbit through the vector ct\ + ^= of the linear isotropy representation of 
a compact symmetric pair with restricted root system of type G 2 (Proposi- 
tion org). 

Remark 5.2. In the case where the rank of the symmetric pair is equal to two, any 
principal orbit of s -representations is an isoparametric hypersurface in the hyper- 
sphere. The family of isoparametric hypersurfaces has a unique minimal isopara- 
metric hypersurface. The theorem above shows some of minimal isoparametric hy- 
persurfaces are austere, furthermore weakly reflective, and some of them are not. 

Before giving a proof of Theorem 15.11 we shall provide some preliminaries. Let 
(G,K) be an irreducible compact symmetric pair. We shall use the notations of 
previous sections. From Lemma [3.41 for a normal vector £ £ T^(Ad(K)H ), the 
shape operator of Ad(K)H in the hypersphere S is given by 

(5.2) (A ( (X*),Y*) = (/i(X*,F*U) = ([Y,[X,H]],Q = —{[X, H], [Y, £]). 

For simplification, we discuss a normalization of a normal vector £. From Lemma [3.4[ 
there exists k G Z% such that Ad(fc)£ e H 1 - n a. Then 

(At(X*),Y*) = (Ad(k)h(X*,Y*),Ad(k)0 

= {h(Ad(k)X*,Ad(k)Y*),Ad(k)0 

= (A Adm Ad(k)X*,Ad(k)Y*) 

= (Ad(fc)- 1 A Ad(fc)c (Ad(fc)X*),r*). 

Thus we have 

A € = M{k)- 1 A AA{k)i M(k). 
This implies that eigenvalues of A Ad ^ k ^ and their multiplicities coincide with those 
of A%. Hence, in order to show whether an orbit Ad(K)H is austere, it suffices to 
check eigenvalues of A^ for £ € H 1 - n o. Hereafter we assume that £ £ H 1 - n a. 
From Lemmas 13.11 and 13.41 

{T a I a e R + — R$} 
is an orthonormal basis of TH{Ad(K)H). For a,0 G R+ — R± we have 
(M(SZ)h),(S* )h) = (a,H){p,H){At{T a ),Tp). 
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On the other hand, from (|5.2| . we have 

(A 6 ((S* a ) H ), (S* ) H ) = -{{S a , H], [Sp, £]) = -(a, H)(J3, £><W- 



Therefore we have 



This shows that T a is an eigenvector of and its eigenvalue is 

M 

(a,H)' 

Hence Ad(K)H is an austere submanifold in S if and only if, for any £ S n a, 
the set 



is symmetric (concerning multiplicities) by the multiplication of — 1. We shall 
describe this condition in terms of a finite subset of a Euclidean space. 

Let A be a finite subset of a Euclidean space V. We consider a condition that, 
for any ueV, 

{(a,v) \ aeA} 

is symmetric (concerning multiplicities) by the multiplication of —1. This condition 
is equivalent to a condition that A is symmetric by the multiplication of —1 on V. 
Indeed, it is obvious that {(a,v) \ a € A} is symmetric whenever A is symmetric. 
Conversely, fix an arbitrary a € A. From the assumption we have 



V= \J{veV | (a,v) =-{b,v}}. 



beA 

If —a ^ A, then the right hand side consists of finite union of hyperplanes of V. 
This is a contradiction. Therefore —a £ A. Consequently A is symmetric by the 
multiplication of -1 on F. 

Let ph '■ a — > H fl a denote the orthogonal projection. An orbit Ad(K)H is 
austere in S if and only if the set 



Ph{o) 
(a.H) 



ae R + - R$ 



is symmetric (concerning multiplicities) by the multiplication of —1. By this crite- 
rion, we can easily see that orbits listed in Theorem 15 . 1 1 are austere submanifolds in 
the hypersphere S. Hereafter we shall prove that all austere orbits can be obtained 
in Theorem 15. II 

We set RR = {xa \ x € R, a 6 R}. We have alrady showed that the orbit 
through any element in RR is weakly reflective in the hypersphere, so we consider 
the orbits through elements in a — RR 



Lemma 5.3. For H G a — RR, the orbit Ad(K)H is an austere submanifold in S 

n 



if and only if there exist a mapping f : i?+ — R^ — > — Rt, without fixed points, 



and constants n a ^ 0, e a = ±1 for each a 6 R + — such that 
(5.3) H = n 



a , /(qQ 
Ml Q ll/(«)ll 
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and 
(5.4) 



"///(<») 



Here we denote by m(p) the multiplicity of a restricted root \x. 

Excepting the case where the restricted root system R is of type BC , the equality 
\5.Jj\ is equivalent to m(a) = m(/(a)), moreover fj={R+ — R+) is even and f 2 = 1. 



Proof. The orthogonal projection pn is defined by 
P H {X)=X-^Mh 



(H,H) 



(Xea) 



Therefore Ad(K)H is an austere submanifold in S if and only if the set 



H 



a€R+- R L 4 



a,H) (H, H) 

is symmetric (concerning multiplicities) by the multiplication of — 1. In other words, 
there exists a mapping / : R + — R+ — > R + — R*t which satisfies 

/(«) 



(5.5) 
and 



H 



(f(a),H) (H,H) 



a if 

{a,H) + (H,H) 

if a 



H 



veR+- R±, 



(u,H) 



(H,H) 

if 
(H,H) 



(a,H) 
/(«) 



if 



(/(a), H) 

for any a € R + — R+. This condition for the multiplicities is equivalent to (15.4 
From (|5.5p . if / has a fixed point a, then if € i?f?. Thus / has no fixed points. 

If we assume (|5.5p . then there exist non-zero real numbers x,y so that if ■ 
xa + yf{a). Applying this to the equation (|5.5p . we have a quadratic equation 

||/(a)||V = IM|V 



with respect to x and y. Thus we have 
hence H can be expressed as 



/(«)« 



if = ira ± 



■a;/ (a) = x||a|| ^r— tt ± 



/(«) 



IN ll/(«)ll 



Since this equality holds for any a G R+ — R+, we have the condition (|5.3|) . 



Replacing a in 
(5.6) 



by /(a), we have 
/ 2 («) /(«) 



2if 



(P(a),H) (f(a),H) (H, H) 



From equations (|5.5[) and (|5.6|) . we have 

a / 2 ( a ) 



(a,fl) (P(a),H)- 
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The above discussion stands for any restricted root systems R, including of type 
BC. Henceforth we assume that R is not of type BC. Then a is the only element 
of R + - R$ which is a scalar multiple of a. Thus f 2 (a) = a. Since / has no fixed 
points, #(R+ — R+) is even. This completes the proof. □ 

Proposition 5.4. In the case where R is of type A\, an austere orbit is one of the 
following except orbits through a restricted root vector: 

(1) when I = 2, the orbit through H = 2e± — e2 — e 3 or e\ + e 2 — 2e 3 , 

(2) when I — 3, the orbit through H = e\ + e 2 — e 3 — e±. 

Proof. In the case of R = Ai, R + is given by R + = {ei — ej | i < j}. Since all 
restricted roots have constant multiplicities, the condition (|5 .4[) of Lemma [5.31 is 
always satisfied. From Lemma 15. 3[ without loss of generalities, we may assume 
that H = ±(a positive root) ± (a positive root). Moreover since any root can be 
translated to e\ — e 2 by the action of the Weyl group, we may assume that H = 
(ei — e 2 ) ± (a positive root). The positive root in the second term of H is one of 
ei - e, (3 < i), e 2 - Cj (3 < i),e l - ej (3 < i < j). 

In the case of H = (ei — e%) ± (ei — e,) (3 < i), ei can be translated to e$ by the 
action of an element of the Weyl group which fixes both e\ and e 2 . Therefore we 
can put 



H = (ei - e 2 ) ± (ei - e 3 ) 



(ei - e 2 ) + (ei - e 3 ) = 2ei - e 2 - e 3 
(ei - e 2 ) - (ei - e 3 ) = -e 2 + e 3 (root) 



Similarly, in the case of H — (ei — e 2 ) ± (e 2 — e^) (3 < i), we can put 
H = (ei - e 2 ) ± (e 2 - e 3 ) = 



d - 2e 2 + e 3 ~ ei + e 2 — 2e 3 
ei — e 3 (root) 

Here, for B.\,H% £ a, we express Hi ~ _ff 2 when i/i can be translated to i/ 2 by 
some element of K. In other words, Hi is equivalent to i? 2 under the action of the 
Weyl group. 

In the case of H = (ei — e 2 ) ± (e^ — ej) (3 < i < j), there exists an element of 
the Weyl group which fixes ei,e 2 and translates to e 3 and ej to e4. Therefore 
we can put 

ei + e 3 - e 2 - e 4 
ei + e 4 — e 2 — e 3 

By the action of the Weyl group, these vectors are equivalent to ei + e 2 — e 3 — e 4 . 
Consequently, it suffices to consider orbits through 



H = (ei - e 2 ) ± (e 3 - e 4 ) 



if = 2ei - e 2 - e 3 , ei + e 2 - 2e 3 , e\ + e 2 - e 3 - e 4 , 

which have a possibility to be austere. 

In the case of H = 2e% — e 2 — e 3 , the only possibility to be the form H = 
(a positive root) ± (a positive root) is H = (ei — e 2 ) + (ei — e 3 ). Thus, from 
Lemma 15.31 the set i?+ — i?^ must be 

R + - R+ = {ei - e 2 , ei - e 3 }. 

When I > 3, since (e 3 — e 4 ,i?) ^ 0, we have e 3 — e 4 S i?+ — f?+. This is a 
contradiction. Hence I = 2 and then Ad(K)H is austere in S. Similarly, the orbit 
through H — ei + e 2 — 2e 3 is also austere. 
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In the case of if = ei+e2— e 3 — e 4 , possibilities to be the form if = (a positive root)± 
(a positive root) are 

if = (ei - e 3 ) + (e 2 - e 4 ) = (ei - e 4 ) + (e 2 - e 3 ). 

Thus f? + — must satisfy 

fl+ - R+ C {ei - e 3 , e 2 - e 4 , ei - e 4 , e 2 - e 3 } 

When / > 4, since (e 4 — es,ff) ^ 0, we have e 4 — es S f£+ — f?+. This is a 
contradiction. Hence I = 3, and then Ad(K)H is austere in S. □ 

Proposition 5.5. In the case where R is of type D\, an austere orbit is one of the 
following except orbits through a restricted root vector: 

(1) the orbit through if = e\, 

(2) when I = A, the orbit through H = e\ + e 2 + e 3 + e 4 , e\ + e 2 + e 3 — e 4 . 

Proof. In the case of i? = Z?;, i? + is given by i?+ = {e^ ± e j | i < j}. Since all 
restricted roots have constant multiplicities, the condition ()5.4|1 of Lemma [5.31 is 
always satisfied. It is easy to see that the orbit through e 4 (or its scalar multiple) is 
austere. Therefore we consider other orbits. From Lemma l5.3[ we can assume H = 
±(a positive root) ± (a positive root). Since any root can be translated to e\ + e 2 
by the action of the Weyl group, we can assume H = (e 4 + e 2 ) ± (a positive root). 
Furthermore any root can be translated to one of 

ei ± e 2 , e 4 + e 3 , e 2 + e 4 , e 3 + e 4 , e 3 - e 4 

by the action of elements of the Weyl group which fix e\, e 2 . Therefore H is one of 
the following: 

H = (ei + e 2 ) ± (ei — e 2 ) = 2ei, 2e 2 ~ 2ei, 

2ei + e 2 + e 3 , 
e 2 - e 3 (root), 

2e 2 + ei + e 4 — 2ei + e 2 + e 3 , 
ei — e 4 (root), 

-ff = (ei + e 2 ) ± (e 3 + e 4 ) ~ ei +e 2 + e 3 + e 4) 
if = (ei + e 2 ) ± (e 3 - e 4 ) - ei + e 2 + e 3 - e 4 . 
Consequently, it suffices to consider orbits through 

H = 2ei + e 2 + e 3 , ei + e 2 + e 3 + e 4 , ei + e 2 + e 3 — e 4 

which have a possibility to be austere. 

In the case of if = 2ei + e 2 + e 3 , the only possibility to be the form H = 
(a positive root) ± (a positive root) is 

if = (ei +e 2 ) + (ei +e 3 ). 

Thus f?+ - f?f must be 

iJ + - R + = {ei + e 2 , e 4 + e 3 }. 

Since (ei — e 2 , if) ^ 0, we have ei — e 2 6 f? + — i?f^. This is a contradiction. Hence 
this orbit is not austere. 

In the case of if = ei+e 2 +e 3 +e 4 , possibilities of the form if = (a positive root)± 
(a positive root) are 

if = (ei + e 2 ) + (e 3 + e 4 ) = (ei + e 3 ) + (e 2 + e 4 ) = (e 4 + e 4 ) + (e 2 + e 3 ). 



if = (ei+e 2 )±(ei+e 3 ) 
if = (ei+e 2 )±(e 2 +e 4 ) 
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Thus i?+ — must satisfy 

R+ - R+ C {ei + e 2 , e 3 + e 4 , e x + e 3 , e 2 + e 4 , ei + e 4 , e 2 + e 3 }. 

When Z > 5, since (e 4 + e5,H) ^ 0, we have e 4 + e 5 e i? + — R^t. This is a 
contradiction. Hence I = 4, and then the orbit Ad(K )H is austere in S. In the case 
of H = e\ + e 2 + e 3 — e 4 , similarly we have I = 4, and then Ad(K)H is austere. □ 

Proposition 5.6. In the case where R is of type P>i,Ci or BCi, an austere orbit 
except orbits through a restricted root vector is the following: 

When R = £? 2 where the multiplicities of the restricted roots are constant, the 
orbit through 

ei +e 2 
H = C1 + — 
is austere. This orbit is a principal orbit. 

Remark 5.7. In the case of R = -B 2 , there exist two singular orbits and these are 
not isometric. Hence from Proposition ^. 91 a principal austere orbit in Proposition 
15.61 is not a weakly reflective submanifold. 

Proof. First we consider the case of R = £>;, where -R+ = {e^, ej ± ej | i < j}. 
From Lemma 15.31 we can assume 

H = lTJi ± mi 
Nl WPW 

i) When a and /3 are both short roots, we can put H = a ± /3. Furthermore, 
since any short root a can be translated to e\ by the action of the Weyl group, we 
can assume H = e 4 ± j3. If (3 = e 4 , then iJ = 2ei and this is equivalent to the orbit 
through a root vector. If (3 = e 3 (j > 2), then H = e\ ± e^- is a root vector. 

ii) When a and are both long roots, we can put H = a ± /3. Since any long 
root a can be translated to e\ + e 2 by the action of the Weyl group, we can assume 
H = (ei + e-x) ± /3. Furthermore /3 can be translated to one of 

P = ei ± e 2 , ei + e 3 , e 2 + e 3 , e 3 + e 4 

by the action of elements of the Weyl group which fix e\ and e 2 . 
In the case of (3 = e\ ± e 2 , i? is equivalent to a root or zero vector. 
In the case of /3 = e\ + e 3 , 



Zf = (e 4 + e 2 ) ± (ei + e 3 ) 



2e 4 + e 2 + e 3 , 
e 2 — e 3 (root). 



When H — 2ei + e 2 + e 3 , the only possibility to be the form H — (a positive root) ± 
(a positive root) is H — (ei + e 2 ) + (ei + e 3 ). Thus i? + — i?^ must be R + — i?^ = 
{e\ + e 2 , ei + e 3 }. On the other hand, since {e\,H) ^ 0, we have e\ G i? + — 
This is a contradiction. Hence this orbit is not austere. 
In the case of (3 = e 2 + e 3 , 



i? = (e 4 + e 2 ) ± (e 2 + e 3 ) 
In the case of f3 = e 3 + e 4 , 
i? = (ei + e 2 ) ± (e 3 + e 4 ) = 



ei + 2e 2 + e 3 — 2ei + e 2 + e 3 , 
e i — e 3 (root). 



ei + e 2 + e 3 + e 4 , 

ei + e 2 — e 3 — e 4 ~ e\ + e 2 + e 3 + e 4 . 
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In this case, 

H = (ei + e 2 ) + (e 3 + e 4 ) = (e x + e 3 ) + (e 2 + e 4 ) = (ei + e 4 ) + (e 2 + e 3 ) 

are possibilities to be the form if = a ± /3. Thus f?+ — f?^ must satisfy 

R+ - R+ C {ei + e 2 , e 3 + e 4 , e x + e 3 , e 2 + e 4 , ei + e 4 , e 2 + e 3 }. 

On the other hand, since (e u H) ^ 0, we have ei G R+-R$. This is a contradiction. 
Hence this orbit is not austere. 

iii) When a is a short root and j3 is a long root, we can assume a = e\ and 

if = e\ ± — p where /3 = ei + e 2 , e 2 + e 3 . 

In the case of if = ei + ei J^ 2 , if Z > 3, then e 3 G f? + — R+. On the other hand, 
there is no fi G f?+ such that 

if = n { e 3 ± ^ 



Thus we have I = 2. In this case R+ — R+ = {ei, e 2 , ei + e 2 , ei — e 2 }. If we define 
/ : R+ - f?f -» fi + - i?f by 

/(ei) = ei+e 2 , /(e 2 ) = e\ - e 2 , 

then if satisfies the condition (|5.3|) of Lemma [5731 Hence this orbit is austere if the 
multiplicities of the restricted roots are constant. 
In the case of if = e\ — ei ^ 2 , we can express if as 

ff = -7i7l{( 1 + 7l) e3 -7f' 

Permuting e\ and e 2 by the action of the Weyl group and replacing e 2 i— > — e 2 , we 
have that this orbit is equivalent to the orbit through 

tt , ei +e 2 

In the case of (3 = e 2 + e 3 , 

rr . e 2 + e 3 e 2 + e 3 

In this case e 3 G i?+ — i?^. On the other hand, there is no fx G R+ such that 

H = n ( e 3 ± -p- 

Hence this orbit is not austere. 

Second we consider the case of R = Ci, where R + = {2ei, e, ± ej \ i < j}. 
For this purpose we shall use the dual mapping and transfer the result of the case 
R = Bi by the dual mapping. A mapping 

is called a dual mapping. This maps a root system to a root system, more precisely, 
a long root is moved to a short root and a short root is moved to a long root. Root 
systems of type Bi and Ci are dual by this mapping, and other irreducible root 



WEAKLY REFLECTIVE SUBMANIFOLDS AND AUSTERE SUBMANIFOLDS 



21 



systems are self-dual. If there exists / which satisfies (|5.3|) for H 6 a, then there 
exists /* which satisfies ()5.3|) for H* . 

In the above discussion, in the case of R — Bi (I > 3), we showed that there are 
no austere orbits except orbits through a restricted root vector. Thus we also have 
that there are no austere orbits except orbits through a restricted root vector in 
the case of R = d (I > 3). When I = 2, C 2 = B 2 . 

Finally we shall consider the case of R = BCi, where R+ = {e,, 2ej, e^ie^ \ i < 
j}. From Lemma 15.31 we can put 

a 

Nl 11/311 

If ||a|| = \\0\\, then we can put H = a ± 0. When a and are both short roots or 
both long roots, H is a scalar multiple of a root vector. When a and are both 
middle roots, we can assume a = e\ + e 2 and 

H = (e 1 + e 2 ) ±0 (0 = ei + e 3 , e 2 + e 3 , e 3 + e 4 ). 

By the action of the Weyl group, these are equivalent to 

H = 2ei + e 2 + e 3 , = ei + e 2 + e 3 + e 4 

or a restricted root vector. In the case of H = 2ei + e 2 + e 3 , we have I = 3 and 

R+- R+ = {ei, e 2 , e 3 , 2ei, 2e 2 , 2e 3 , e\ ± e 2 , ei ± e 3 , e 2 + e 3 }. 

Since there is no / which satisfies (|5.3[) . this orbit is not austere. In the case of 
H = ei + e 2 + e 3 + cj, we have Z = 4 and 

R+-R+ = {ei,2 ei | 1 < i < 4} 

U{ei -I- e 2 , ei + e 3 , ei + e4, e 2 + e 3 , e 2 + a, e 3 + e4}. 



Since there is no / which satisfies (|5.3p . this orbit is not austere. 

It remains the case where ||a|| < \\0\\- When a is a short root and is a long 
root, H is a scalar multiple of a root vector. By the dual mapping, we can identify 
two cases, where a is a short root and is a middle root, and where a is a middle 
root and is a long root. Therefore we shall discuss the former. In this case, we 
can assume a = e\ and 

H = ei ± —p where = e\ + e 2 , e 2 + e 3 . 

Similarly with the case of the restricted root system of type B, 

u , e i + e 2 „- , e 2 + e 3 

H = ei H = — , -H = ei H ^ — 

V2 \/2 

have a possibility to be austere. When H = e\ + e2 ^ 3 , we have I = 3 and 
R + - R+ = {ei, e 2 , e 3 , 2ei, 2e 2 , 2e 3 , ei ± e 2 , ei ± e 3 , e 2 + e 3 }. 



Since there is no / which satisfies (I5.3p . this orbit is not austere. When H = 
e\ + ei ^ 2 , the orbit has a possibility to be austere if I = 2. In this case, the orbit 
is a principal orbit. This orbit is austere if the sum of the multiplicities of long 
roots and short roots coincides with the multiplicity of middle roots. From the 
classification of symmetric pairs, there does not exist such a symmetric pair. □ 
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Proposition 5.8. In the case where R is of type G 2 , the orbit through 

H = ai H -= 

is the only austere orbit except orbits through a restricted root vector. This orbit is 
a principal orbit. 



Remark 5.9. By the same discussion in Remark 15.71 this principal austere orbit 
is not a weakly reflective submanifold from Proposition ^. 91 

Proof. The fundamental system F of the restricted root system of type G 2 is given 
by F = {a>i = e\ — e 2 , a 2 = — 2ei + e 2 + e 3 } and the set R + of positive roots is 

R = f u{ Q i + Q 2 = _e l + e 3, 2ai + a 2 = -e 2 + e 3 , 
+ [_ 3ai + a 2 = ei — 2e 2 + e 3 , 3ai + 2a 2 = —e\ — e 2 + 2e 3 

In the case of G 2 , since all restricted roots have constant multiplicities, the condition 
(|5.4[) of Lemma 15.31 is always satisfied. From Lemma 15.31 we can put 

H = U\ ± M\ {a ^ €R+) - 

When a and (3 are both short roots, we can put H = a±(3 (a 7^ (3). Furthermore, 
since any short root a can be translated to ot\ by the action of the Weyl group, we 
can assume 

H = ai ± (3 where (3 — ot\ + a 2 , 2ai + a 2 - 
In the case of (3 — a\ + a 2 , 

H = ol\ ± (ai + a 2 ) = 2ai + a 2 , ~a 2 . 
Then H is a root vector. In the case of (3 — 2ct\ + a 2 , 

H = ol\ ± (2ai + a 2 ) — 3«i + a 2 , — ai — a 2 . 

Then H is a root vector. 

When a and /3 are both long roots, we can put H = a ± [3 (a ^ (3). Since any 
long root a can be translated to a 2 by the action of the Weyl group, we can assume 

H = a 2 ± (3 where (3 = 3cni + a 2 , 3ai + 2a 2 . 
In the case of (3 = 3ai + a 2 , 

H = a 2 ± (3ai + a 2 ) = — 3ai, 3«i + 2a 2 . 
Then is a scalar multiple of a root vector. In the case of [3 = 3«i + 2a 2 , 

H = a.2 ± (3ai + 2a 2 ) = — 3ai — a 2 , 3(ai + a 2 ). 

Then _ff is a scalar multiple of a root vector. 

When a is a short root and [3 is a long root, we can assume a = ai and 

H = ai± —= where (3 = a 2 , 3ai + a 2 , 3ai + 2a 2 . 
v3 

We note that the orbit though H is a principal orbit. In these cases, H is equivalent 
to a scalar multiple of «i + by the action of the Weyl group. In the case of 
H = a 1 + ^, 

If 3ai + a 2 \ l/o . 3a i + 2a 2 

-H = — ^= ai + a 2 H = — = — = 2ai + a 2 



VZ + 1 V V3 / V3 + 2 V V3 

Thus from Lemma 15.31 this orbit is austere. This completes the proof. □ 
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Proposition 5.10. In the case of R = Fi, there are no austere orbits except orbits 
through a restricted root vector. 

Proof. In this case i?+ is given by 

R+ = {e i }i< i < 4 U {e, ± e i }i<i< J <4 U ji(ei ± e 2 ± e 3 ± e 4 ) 
From Lemma [5JH we can assume 



Nl WW 

When a and /3 are both short roots, we can put H = a ± (3. In this case, since any 
short root can be translated to e\ by the action of the Weyl group, we can put 

H = e\ ± j3 where /3 = (i > 2), i(e! ± e 2 ± e 3 ± e^). 

In the case of (3 = ej, H is a root vector. In the case of (3 = \{e\ ± e 2 ± e 3 ± 64). 

tr 1 1 1 1 \ / ei + i(ei ± e 2 ± e 3 ± e 4 ), 

" = ei ± -(ei ± e 2 ± e 3 ± e4> = < , , , , , , 

2 V I 2"(ei ± e 2 ± e 3 ± e 4 ) (a root). 

Thus we consider the case of H = e\ + \{e\ ±e 2 ±e 3 ± e^). In this case, (e4,H) 7^0, 
however, there does not exist (3 € i?+ such that 

H = n { ei± W\ 

Hence from Lemma 15.31 this orbit is not austere. 

When a and (3 are both long root, we can assume H — e\ + e 2 ± ei ± ej (i < j). 
Moreover, we exclude H which is a scalar multiple of a root vector. Then 

H = 2ei + e 2 ± e u e x + 2e 2 ± a (i = 3, 4). 

The reflection s e3 -e 4 permutes e 3 and e4, and fixes e%, e 2 . Therefore we can put 

H = 2ei + e 2 ± e 3 . 

In this case, (e 3 , i?) 7^ 0, however, there does not exist (3 £ R+ such that 

H = n { e3± W\ 

Hence this orbit is not austere. 

When a is a short root and (3 is a long root, we can put 

H = ei+ (Kj). 

Moreover, by the action of the Weyl group we can assume 

±d + e 2 e 2 + e 3 
H=e\-\ = — or H = e\-\ =— . 

In the case of H = e\ + e2 J^ 3 , we have (e 3 , H) 7^ 0. However, there does not exist 
(3 G R + such that 

/3 



ff = n,e3± PI 
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Hence this orbit is not austere. In the case of H = ei + ±e ^t e2 , we have (e 2 
e 3 , if) 0. However, there does not exist (3 £ R+ such that 

'e 2 + e 3 /? 



Hence this orbit is not austere. 



V2 



□ 



Proposition 5.11. In the case of R — E%, there are no austere orbits except the 
orbits through a restricted root vector. 

Proof. In the case of R = E s , R + is given by 

/» = { ±< , + < , i : i< J - *} uU(e8+^(-ir (i) ei) £i/(*)e2z| 

I i=l i=l J 



Since all restricted roots have constant multiplicities, the condition (|5.4[) of Lemma l5.3 
is always satisfied. From Lemma 15.31 we can put H = e\ + ei + (3 where 

±ei ± e 2 , 

±ei±e; (3<«<8), 

/? = <( ±e 2 ±e l (3 < i < 8), 

ie.iej (3<i<j<8), 

I 5Er=i(-ir w ei (EL"(i)€E2Z). 

i) In the case of /3 = ±ex ± e2, 

ei + e 2 + ei + e 2 = 2(ei + e 2 ) (twice of a root), 
ei + e 2 + ei — e 2 = 2ei, 
ei + e 2 - ei + e 2 = 2e 2 , 
e\ + e 2 — ei — e 2 = 0. 



if 



When H = 2e u 
R — R A = {±ei±< 



2<i< 




i=2 



H = 



Then there does not exist j3 € f? + such that iJ = n-(iX^=i ± /3). Hence this 
orbit is not austere. When H = 2e 2 , since the reflection s ei _ e2 permutes ej and 
e 2 , this orbit is equivalent to the orbit through H = 2e x . Hence this orbit is not 
austere. 

ii) In the case of (3 = ±ei ± e, (3 < i < 8), 

2ei + e 2 ± ej, 
e 2 ± ej (a root). 

The reflection s e3 - ei (i > 4) fixes ei,e 2 and permutes e3 and ej. Thus the orbit 
through = 2ei + e 2 ±ei is equivalent to the orbit through H = 2ei+e 2 ±e3. Then 
(ei+ei, H) ^ 0, however, there does not exist (3 £ R+ such that H = n(ei+e4±/3). 
Hence this orbit is not austere. 

iii) In the case of (3 = ±e 2 ± (3 < i < 8), 

ei + 2e 2 ± 
e\ ± ei (a root). 

By the action of the Weyl group, the orbit through H — e\ + 2e 2 ± ei is equivalent 
to the orbit through H = 2ei + e 2 ± e^. Hence this orbit is not austere. 



H 
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iv) In the case of (3 = ±a ± ej (3 < i < j < 8), we can assume 

H = ei + e 2 ± e3 ± e4. 

Then (ei + e 5 ,H) ^ 0, however, there docs not exist (3 e i? + such that H 
«(ei + ± (3). Thus this orbit is not austere. 

v) v-1) In the case of 

8 8 

P = \{-e, -e 2 + ^(-l)^) e .) 5 "(0 e 

i—3 i— 3 

then 

1 8 
ff= T(ei + e 2 + ^(-l)"W ei ) (a root). 



2 

v-2) In the case of 

1 8 8 

= 2 (-ei + e 2 + Y,{-l)"®ei), "W e 2Z + 1, 

i=3 i=3 

then 

H = \{e, + 3e 2 + £(-1)"^) ~ ±(3 ei + e 2 + £(-l)"(0 e .). 

i=3 i=3 

v-3) In the case of 

. 8 8 

(3 = -(ei - e 2 + ^(-lJ-Wgi), ^ e 2Z + 1, 

i=3 i=3 

then 

F=i(3e 1 + e2 +^(-irW ei ). 

i=3 

In this case, there exists an i such that u(i) = 1. Permuting ei and e3 by the action 
of the Weyl group, we have 

1 8 8 

H = -(3ei + e 2 - e 3 + ^(-l) 1 ^), £ € 2Z. 

Then (e 2 — es,H) ^ 0, however, there does not exist /3 e J? + such that H = 
n(e 2 — e 3 ± /3). Thus this orbit is not austere, 
vi) In the case of 

8 8 

13 = l( ei + e 2 + ^(-l)"W ei)j £ e 2Z, 

i= 3 i= 3 

then 



H= J(3 ei +3e 2 + ^(-irW ei ). 



2 

i=3 

In this case (ei + es,H) ^ 0, however, there does not exist /3 G R + such that 
i? = n(ei + e 3 ± /?). Thus this orbit is not austere. □ 

Proposition 5.12. In the case of R — Ei, there are no austere orbits except the 
orbits through a restricted root vector. 
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Proof. In the case of R — Er, all restricted roots have constant multiplicities. Thus 



the condition (15. 4p of Lemma \5. 31 is always satisfied, a — {Ei=i£i e i 
and 



From Lemma 



= {±e 

U 



• ej I 1 < i < j < 6} U {e 8 - e 7 } 



6 

E 



v(i) e 2Z + 1 



we can assume H = e 7 

/3 = 



3 + /? where 

±ei±ej (1 < « < j < 6), 
iKer-es + EliC-ir^ei). 

In the case of /3 = ±e^ ± (1 < i < j < 6), we take k such that 1 < k < 6 and 
7^ i, j. Then ej + efe G i?+ and (H,ej + efe) 7^ 0, however, there does not exist 
a G i? such that iJ = n(ej + + a). Thus this orbit is not austere. 
In the case of /3 = ±±(e 7 - e 8 + £f =1 (-l)" (i) ei) then 

f KSey-Ses+ELiC-irWei), 

1 ^-eg-ELiM)^) (a root). 

Therefore it suffices to consider the case of H — ^(3e 7 — 3es + Ei=i("l)"''' e ')- ^ n 
this case, either e\ + e2 or ei — e2 is an element of R+ — J?+. Denote this element 
by a. Then there does not exist [3 G i?+ such that H = n(a ± [3). Hence this orbit 
is not austere. This completes the proof. □ 

Proposition 5.13. In the case of R — Eq, there are no austere orbits except orbits 
through a restricted root vector. 

Proof. In the case of R = Eq, all restricted roots have constant multiplicities. Thus 



{± 



the condition 
-£s} and 

R+ = 



of Lemma [5~3l is always satisfied, a = {J2i=i £i e i I ^6 = £7 
ej I 1 < i < j < 5} 



U 



.(e 8 - e7 - e6+ ^(-l)-W e .) 



5 

E 

i=\ 



v{%) e 2Z 







From Lemma |5.3[ we can assume H = e\ + e 2 + (3 where 

±(ei - e 2 ), 

±e 2 ±ei (3<i< 5), 
±e l ±e 3 (3<i<j< 5), 

[ ±±(e 8 - e 7 - e 6 + J2Li(-lY {l) ^)- 

i) In the case of (3 = ±(ei — 62), then _ff = 2e\, 2e 2 . For 

q = 7^(e 8 - e 7 - e 6 + ei + e 2 + e 3 - e 4 - e 5 ) € R + - R.f 



there does not exist (3 £ R+ such that H = n(a±(3). Thus this orbit is not austere, 
ii) In the case of (3 = ±e 2 ± ei (3 < i < 5), then 



i? = ei + e 2 ± e 2 ± e l — 



e\ + 2e 2 ± ei, 
e\ ± ei (root). 
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Therefore it suffices to consider the case of H = ei + 2e 2 ± ei. In this case, for j 
with 3 < j < 5 and j ^= i, we have ei + ej G i?+ — i?^. However, there does not 
exist (3 G i?+ such that = n(e\ + ej ± (3). Thus this orbit is not austere. 

iii) In the case of (3 = ie^ ± ej (3 < i < j < 5), then H = ei + e2 ± ej ± ej. For 
A: with 3 < £; < 5 and A; ^ we have ei + G i?+ — . However, there does 
not exist (3 G i?+ such that H = n{e\ + ± (3). Thus this orbit is not austere. 

iv) In the case of 

5 5 

/3 = ±-(e 8 -e 7 - e6 + ^(-ir«e,) where £i/(i)e2Z, 



(=i 



then 



Here 



H = ei+e 2 ±-( 



e 7 - e 6 



£(-ir 



are root vectors. 
In the case of 



ei + e 2 + - 



-e 7 -e 6 -e 1 -e 2 + ^(-l) 1 'We i ), 

I 5 

ei + e 2 - -(e 8 - e 7 - e 6 + ei + e 2 + ^(-l) l,(l) e i ) 



i=3 



ei + e 2 + - 



# = < 



- e 7 - e 6 + ei - e 2 + ^(-lf^e*), 

z=3 

1 5 

ei + e 2 + -(e 8 - e 7 - e 6 - e x + e 2 + y^(-l)" w ej), 

i=3 

1 5 
ei + e 2 - -(e 8 - e 7 - e 6 + e x - e 2 + ^(-l)^ W ei), 

i=3 

1 5 

- e 7 - e 6 - ex + e 2 + ^(-lf^e;), 



ei + e 2 



i=3 



we have (e\ — e 2 ,i/) 7^ 0. However there does not exist /3 G R+ such that if 
n(ei — e 2 ± Thus this orbit is not austere. 
Finally, in the case of 



1 5 
si + e 2 + -(e 8 - e 7 - e 6 + ei + e 2 + ^(-lf^e;), 



= < 



i=3 



f'l + ('2 



e 7 - e6 -e 1 -e 2 +^(-l) l '«e i ) 



i=3 



we have (ei + e3,H) ^ 0. However there does not exist (3 G i?+ such that H = 
n(ei + e$ ± /3). Thus this orbit is not austere. This completes the proof. □ 



By discussions above, we completed the proof of Theorem 15.11 and Theorem 14. II 
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6. Miscellaneous results 

In this section, we shall concern with some results on weakly reflective subman- 
ifolds besides orbits of s-representations. 

Proposition 6.1. Let M\ and M 2 be weakly reflective submanifolds in Riemannian 
manifolds M\ and M 2 , respectively. Then Mi x M 2 is a weakly reflective subman- 
ifold in Mi x M 2 . 

Proof. We take (xi , x 2 ) £ M 1 x M 2 and a normal vector (£i , £ 2 ) e (Mi x M 2 ). 

Let (7{ 1 and a^ 2 be reflections of Mi and M 2 in Mi and M 2 with respect to £i and 
£ 2 , respectively. Then cr^ 1 x cr^ 2 is an isometry of Mi x M 2 and satisfies 

0?i x ^2)^1.^2) = (xi,x 2 ), 

d(c7 6 x o- 6 ) (xi>X2) (^,^ 2 ) = -(6,6), 

(er 6 x CT| 2 )(Mi x M 2 ) = Mi x M 2 . 

Thus Mi x M 2 is a weakly reflective submanifold in M x M 2 . □ 

The following proposition states that the cone over a weakly reflective subman- 
ifold in a sphere is also a weakly reflective submanifold in a Euclidean space. 

Proposition 6.2. Let M be a weakly reflective submanifold in a unit sphere 5 n_1 (l). 
Then the cone 

C(M) = {tx\t£B., t>0, x£ M} 
over M is a weakly reflective submanifold in a Euclidean space R" . 

Proof. Fix x £ M. We note that for arbitrary t £ R (t > 0), we have 

T t i(C(M)) =T^M cT x S n -\l). 

For £ £ T^M, a reflection of M with respect to £ satisfies 

ct £ (x) = x, {da ( ) x (, = -f, **(M) = M. 

Since cr £ is an isometry of S" l_1 (l), it can be expressed as an orthogonal matrix. 
Thus <j£ acts on R n and satisfies 

crj(te) = to-^(x) = tx, (da^) x £, = 

In addition, for a;' e M, i' e R, t' > 0, we have 

a^t'x') =t'a s (x') £ C(M). 

Therefore a^(C(M)) — C(M). Hence C(M) is a weakly reflective submanifold in 
R". □ 

Next we shall describe the relationship between weakly reflective submanifolds 
in an odd dimensional sphere and in a complex projective space. 

Proposition 6.3. Let M be a weakly reflective submanifold in the hypersphere S C 
C n+1 . If M is invariant under U(l)-actions and if a reflection of M with respect 
to each normal vector is a unitary action, then the image P(M) of M is a weakly 
reflective submanifold in CP n , where P is the natural projection P : S — > CP". 
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Proof. By the definition of P, T x (U(l)x) = kerdP x at each point x € S. Thus 
dP x : T x (U(l)x) — > T x (CP n ) is an isometric linear isomorphism. Since M is in- 
variant under the [/(l)-actions, P(M) is a submanifold in CP™. Moreover, since 
T x (U(l)x) C T S M for x G M, we have T^([/(l)x) D T^M. Thus dP, : T^M -> 
T^ x j(P(M)) also gives an isometric linear isomorphism. Let cr^ denote a reflection 

of M with respect to £ € T^M . From the assumption, er^ is a unitary transfor- 
mation of C" +1 . Hence er^ induces an isometry of CP™. Since cr^(x) = x and 
a 6 (M) = M, we have cr ? (P(x)) = P(x) and ct 5 (P(M)) = P(M). In addition we 
have 

= rfP x (7 S (0 - dP a (-0 = -dP x (0- 

Thus <7£ is a reflection of P(M) with respect to a normal vector dP x (£) at P(x). 
Hence P(M) is a weakly reflective submanifold in CP™. □ 

Corollary 6.4. .An or&it of the s -representation of an irreducible compact Her- 
mitian symmetric pair through a restricted root vector induces a weakly reflective 
submanifold in a complex projective space. 

Proof. The center of the linear isotropy subgroup of an irreducible compact Her- 
mitian symmetric pair is U(l) ( f> ]). Thus all orbits are invariant under U(l). 
Furthermore an orbit in the hypersphere S through a restricted root vector is a 
weakly reflective submanifold. From the proofs of Lemma [3^41 and Proposition ^. 3[ 
a reflection of an orbit through a restricted root vector with respect to each nor- 
mal vector is a unitary transformation. Hence, from Proposition 16.31 we have the 
conclusion. 

□ 
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